Computer Vision - Project 1

Image representation and preprocessing

TOC

Imports & Utilities

Image Download

1. Image Analysis

Image sizes

a. Initial Observations

Cathedral Set

Airport Set:

b. Images Covering

Changing Region

Changing region weighted

Very bright areas are different between a greater number of images, black areas are the same in all images.

Differences in the sky and shadow changes heavily influence the detected changing region, we will see how it will influence the result.

c. Images Color Density

Mean Channel Values and Image Standard Deviation

d. Images statistics

Mean and median results

For both sets the median image already yields a relatively pure image. We will use it as a reference point further in the comparison.

2. Purification

a. Histogram Method

Description

The first 2 steps of this approach are based on the:
Aseem Agarwala, Mira Dontcheva, Maneesh Agrawala, Steven Drucker, Alex Colburn, Brian Curless, David Salesin, Michael Cohen. Interactive Digital Photomontage. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2004), 2004.

Link

It works in the following way:

Intermediate Steps

Final Results

Advantages

b. Mean Pixel Distance and Channel Intensity Method

Description

This method works by calculating the masks of distances to the mean pixel value that are higher than a given threshold and then merging them with corresponding high channel intensity masks - that is, masks denoting whether in a given pixel any of the color channels stands out more than the other two (is higher than given tolerance). It should very accurately remove entities that are of colors that stand out from the background, or have high intensity of one color - the motivation for the latter being high color intensity of people clothing in the cathedral set.

The exact steps are as follows:

Intermediate Steps

Final Results

Advantages

c. Comparison and tweaks

All results together

The Mean Pixel Distance and Color Channel Intensity Method works better for the Cathedral dataset whilst the Histogram Method is superior for the Airport dataset.

Both methods remove vast majority of the undesired objects.

We can combine obtained results by median:

Or by mean:

d. Bonus - Greedy Method

Description

This aproach is based on greedy method. The distance measure is Chebyshev distance. It works in following way:

Advantages

Drawbacks

Function get pixel

Function get pixels from diffrent pixel and compute distance between them and target pixel. Then return best pixel.

Why Chebyshev distance?

This will be our base so clearly dark blue

Now we can compare basic distances between our base dark blue and pink and Medium Slate Blue (name based on www.color-name.com) | | pink | second | |:------------|:-------------------------------------------:|---------------------------------------------:| |chebyshew | max(|237-0|,|0-0|,|231-231|) = 237 | max(|120-0|,|120-0|,|231-231|) = 120 | |manhattan | (|237-0|+|0-0|+|231-231|) = 237 | (|120-0|+|120-0|+|231-231|) = 240 | |euclidian | sqrt(|237-0|^2+|0-0|^2+|231-231|^2) = 237 | sqrt(|120-0|^2+|120-0|^2+|231-231|^2) = 170 |

The best result is given by chebyshew (pink has big distance and Medium Slate Blue which is still blue has smaller difference)

Function get image

This function generate image based on two paramaters h and v where h maens how far horizontally is the pixel with which we are comparing and v how far vertically. "Images_original" are images on which we based. We compare allways one pixel. In the experience it turned out that the results are much worse when based on a group of pixels. Better results are obtained when we create several images and then assemble them into one. This approach is also less complex computationally.

Result

Obtained results are quite good but images still have horizontal and vertical lines as remains of objects

3. Failed Attempts

a. Edge Weight Method

Description

The basis of the algorithm is the fact that the unconcealed entities usually stand out on their direct background - there is an edge between them. We can exploit that by treating the inverted results of edge detection algorithm as weights - the background edges will appear on every weight map with similar value hence won't be affected, whilst the edges of moving objects should appear on singular images, in which case the algorithm will choose the images in which these edges do not appear, thus removing the entities from the image.

Algorithm works as follows:

This method failed as it does not yield results much better than simple mean and is worse than the median.

The reason being that it only removes the objects on edges and not in their entirety, and does not work iteratively as it smoothes out edges after being applied so it cannot self-improve.

b. Dynamic pixel substituion with edge sum heuristic

Description

The algorithm was based on initial observation that the sum of Canny edge detection result with high thresholds is somewhat correlated with the purity of the image (the lesser the sum the purer the image). It checked each pixel (or group of pixels) whether or not substituting it with a pixel (pixels) from any other pictures improved the result, and it replaced that pixel with the best substitute.

Steps:

[Code lost in the exploration process]

The algorithm was very slow and it assumed that the purity estimation worked good enough on local level, however that was not enough and it yielded mixed (but unsatisfactory) results.